{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# fastai: lesson 1 - Pets example with Amazon SageMaker\n",
    "\n",
    "## Pre-requisites\n",
    "\n",
    "This notebook shows how to use the SageMaker Python SDK to run your fastai library based model in a local container before deploying to SageMaker's managed training or hosting environments.  This can speed up iterative testing and debugging while using the same familiar Python SDK interface.  Just change your estimator's `train_instance_type` to `local`. \n",
    "\n",
    "In order to use this feature you'll need to install docker-compose (and nvidia-docker if training with a GPU).\n",
    "\n",
    "**Note, you can only run a single local notebook at one time.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import io\n",
    "import subprocess\n",
    "\n",
    "import PIL\n",
    "\n",
    "import sagemaker\n",
    "from sagemaker.pytorch import PyTorch, PyTorchModel\n",
    "from sagemaker.predictor import RealTimePredictor, json_deserializer\n",
    "\n",
    "from fastai.vision import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "The **SageMaker Python SDK** helps you deploy your models for training and hosting in optimized, productions ready containers in SageMaker. The SageMaker Python SDK is easy to use, modular, extensible and compatible with TensorFlow, MXNet, PyTorch and Chainer. This tutorial focuses on how to create a convolutional neural network model to train the [Oxford IIIT Pet dataset](http://www.robots.ox.ac.uk/~vgg/data/pets/) as per [Lesson 1 of the fast.ai MOOC course](https://course.fast.ai/videos/?lesson=1) using **PyTorch in local mode**.\n",
    "\n",
    "### Set up the environment\n",
    "\n",
    "To setup a new SageMaker notebook instance with fastai library installed then follow steps outlined [here](https://course.fast.ai/start_sagemaker.html).\n",
    "\n",
    "This notebook was created and tested on a single ml.p3.2xlarge notebook instance. \n",
    "\n",
    "Let's start by specifying:\n",
    "\n",
    "- The S3 bucket and prefix that you want to use for training and model data. This should be within the same region as the Notebook Instance, training, and hosting.\n",
    "- The IAM role arn used to give training and hosting access to your data. See the documentation for how to create these. Note, if more than one role is required for notebook instances, training, and/or hosting, please replace the sagemaker.get_execution_role() with appropriate full IAM role arn string(s)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to test your training or hosting of your fastai model then run the following cell to update the Docker daemon default shared memory to 2gb. Only run this command if you are using the `ml.p3.2xlarge` instance type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! sudo cp daemon.json /etc/docker/daemon.json && sudo pkill -SIGHUP dockerd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sagemaker_session = sagemaker.Session()\n",
    "\n",
    "bucket = sagemaker_session.default_bucket()\n",
    "prefix = 'sagemaker/DEMO-fastai-pets'\n",
    "\n",
    "role = sagemaker.get_execution_role()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the Oxford Pets dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will download the dataset and save locally on our notebook instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = untar_data(URLs.PETS); path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Preview"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path_anno = path/'annotations'\n",
    "path_img = path/'images'\n",
    "\n",
    "fnames = get_image_files(path_img)\n",
    "fnames[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Upload the data\n",
    "We use the ```sagemaker.Session.upload_data``` function to upload our datasets to an S3 location. The return value inputs identifies the location -- we will use this later when we start the training job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inputs = sagemaker_session.upload_data(path=path, bucket=bucket, key_prefix=prefix)\n",
    "print('input spec (in this case, just an S3 path): {}'.format(inputs))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Construct a script for training and inference\n",
    "Here is the full code that both trains the model and does model inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "!pygmentize source/pets.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Script Functions\n",
    "\n",
    "SageMaker invokes the main function defined within your training script for training. When deploying your trained model to an endpoint, the model_fn() is called to determine how to load your trained model. The model_fn() along with a few other functions list below are called to enable predictions on SageMaker.\n",
    "\n",
    "### [Predicting Functions](https://github.com/aws/sagemaker-pytorch-containers/blob/master/src/sagemaker_pytorch_container/serving.py)\n",
    "* model_fn(model_dir) - loads your model.\n",
    "* input_fn(serialized_input_data, content_type) - deserializes predictions to predict_fn.\n",
    "* output_fn(prediction_output, accept) - serializes predictions from predict_fn.\n",
    "* predict_fn(input_data, model) - calls a model on data deserialized in input_fn.\n",
    "\n",
    "The model_fn() is the only function that doesn't have a default implementation and is required by the user for using PyTorch on SageMaker. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a training job using the sagemaker.PyTorch estimator\n",
    "\n",
    "The `PyTorch` class allows us to run our training function on SageMaker. We need to configure it with our training script, an IAM role, the number of training instances, and the training instance type. \n",
    "\n",
    "For local training with GPU, we could set this to ```local_gpu```.  In this case, `instance_type` was set below based on whether you're running a GPU instance. If `instance_type` is set to a SageMaker instance type (e.g. ml.p2.xlarge) then the training will happen on SageMaker.\n",
    "\n",
    "The parameter `data_location` determines where the training data is. If training locally then it can be set to the local file system to avoid having to download from S3. If training on SageMaker then it needs to reference the training data on S3.\n",
    "\n",
    "After we've constructed our `PyTorch` object, we fit it using the data we uploaded to S3. Even though we're in local mode, using S3 as our data source makes sense because it maintains consistency with how SageMaker's distributed, managed training ingests data.\n",
    "\n",
    "If you want to train locally then uncomment out all of the lines in the code block below.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Comment out all lines below if not training locally\n",
    "data_location='file://'+str(path)\n",
    "instance_type = 'local'\n",
    "if subprocess.call('nvidia-smi') == 0:\n",
    "    ## Set type to GPU if one is present\n",
    "    instance_type = 'local_gpu'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to train your model on SageMaker then comment out the cell above and uncomment the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Comment out all lines below if not training on SageMaker\n",
    "#data_location=inputs\n",
    "#instance_type = 'ml.p3.2xlarge'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pets_estimator = PyTorch(entry_point='source/pets.py',\n",
    "                         base_job_name='fastai-pets',\n",
    "                         role=role,\n",
    "                         framework_version='1.0.0',\n",
    "                         train_instance_count=1,\n",
    "                         train_instance_type=instance_type)\n",
    "\n",
    "pets_estimator.fit(data_location)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy the trained model to prepare for predictions\n",
    "\n",
    "First we need to create a `PyTorchModel` object from the estimator. The `deploy()` method on the model object creates an endpoint (in this case locally) which serves prediction requests in real-time. If the `instance_type` is set to a SageMaker instance type (e.g. ml.m5.large) then the model will be deployed on SageMaker. If the `instance_type` parameter is set to `local` then it will be deployed locally as a Docker container and ready for testing locally.\n",
    "\n",
    "First we need to create a `RealTimePredictor` class to accept `jpeg` images as input and output JSON. The default behaviour is to accept a numpy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ImagePredictor(RealTimePredictor):\n",
    "    def __init__(self, endpoint_name, sagemaker_session):\n",
    "        super(ImagePredictor, self).__init__(endpoint_name, sagemaker_session=sagemaker_session, serializer=None, \n",
    "                                            deserializer=json_deserializer, content_type='image/jpeg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to deploy your model locally then comment out the ```instance_type``` declaration below. \n",
    "\n",
    "If you want to deploy your model on SageMaker then uncomment the the ```instance_type``` declaration below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment out for SageMaker Deployment\n",
    "#instance_type = 'ml.c5.large'\n",
    "\n",
    "pets_model=PyTorchModel(model_data=pets_estimator.model_data,\n",
    "                        name=pets_estimator._current_job_name,\n",
    "                        role=role,\n",
    "                        framework_version=pets_estimator.framework_version,\n",
    "                        entry_point=pets_estimator.entry_point,\n",
    "                        predictor_cls=ImagePredictor)\n",
    "\n",
    "pets_predictor = pets_model.deploy(initial_instance_count=1,\n",
    "                                       instance_type=instance_type)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Invoking the endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "urls = []\n",
    "# English Cocker Spaniel\n",
    "urls.append('https://s3.amazonaws.com/cdn-origin-etr.akc.org/wp-content/uploads/2017/11/16105011/English-Cocker-Spaniel-Slide03.jpg')\n",
    "# Shiba Inu\n",
    "urls.append('https://upload.wikimedia.org/wikipedia/commons/thumb/6/6b/Taka_Shiba.jpg/1200px-Taka_Shiba.jpg')\n",
    "# German Short haired\n",
    "urls.append('https://vetstreet.brightspotcdn.com/dims4/default/232fcc6/2147483647/crop/0x0%2B0%2B0/resize/645x380/quality/90/?url=https%3A%2F%2Fvetstreet-brightspot.s3.amazonaws.com%2Fda%2Fa44590a0d211e0a2380050568d634f%2Ffile%2FGerman-Shorthair-Pointer-2-645mk062111.jpg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get a random selection\n",
    "img_bytes = requests.get(random.choice(urls)).content\n",
    "img = PIL.Image.open(io.BytesIO(img_bytes))\n",
    "img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = pets_predictor.predict(img_bytes)\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clean-up\n",
    "\n",
    "Deleting the local endpoint when you're finished is important since you can only run one local endpoint at a time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pets_estimator.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_p36",
   "language": "python",
   "name": "conda_pytorch_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "notice": "Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
